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In finding the adequate way to prioritize proposals, the Brazilian par¬ 
ticipation community agreed about the measurement of two indexes, 
one of approval and one of participation. Both practice and literature 
is constantly handled by the experts involved, and the formalization 
of such model and metrics seems novel. Also, the relevance of this 
report is strengthened by the nearby use of these indexes by the 
Brazilian General Secretariat of the Republic to raise and prioritize 
proposals about public health care in open processes. 
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Online decision making is a kind of recommendation system 
with special appeal for online social participation and elec¬ 
tronic governments. This poses challenges on the design of 
such processes regarding validity, security and the adequate 
indicators. Indeed, the processes themselves vary, and the 
fact that the indexes presented here seem not to be formalized 
and published is an evidence that such online decision making 
is very recent phenomena. 

The main contribution of this report is a modeling for an 
online voting process 0 [HE] with the following characteris¬ 
tics: 


• proposals might be inserted by voters after the voting phase 
started. 

• Voting might be extended as a permanent process. In other 
words, voting on and adding new proposals might be open 
continuously. 

• A proposal is presented to a voter one by one as random 
outcomes of all proposals. 

• Each vote might be of one and only type among: “approve”, 
“disapprove” and “indifferent”. 

• Voters vote without authentication. 

• Intended mostly for national rankings, but can also be local 
or have foreign participation. 

• Should result in a ranking of proposals to assist public man¬ 
agement. 


This setting requires care about security and validity. Some 
of which are: 


Approval and participation indexes. The approval index on and 
the participation index Vi of the proposal i was defined as: 
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where vf, v i and r/i are approval count, disapproval count 
and exhibition count, respectively. Note that on G [—1,1], 
7i G [0,1], and v° = i]i — vf — is the count of the “indiffer¬ 
ent” manifestations received by proposal i. Also, such on and 
7 i indexes are expected, for each proposal i, to be a constant 
plus a sampling estimate error that should be smaller as r/i 
raises. This error is thought to be acceptable if 7]i is above 
a threshold rj established by the participation community and 
public managers. As an initial decision, the staff agreed to use 
rj as to select 10 — 20% of all proposals. A threshold 7 can 
be used as a required level of engagement for proposals to be 
relevant, while the threshold a is used to classify the outcome 


”, “disapproved” 

and “clash”. More specifically: 

Vi > V 


i is sampled 

H > 7 

=> 

i is relevant 

|a/ < a 

=> 

i is a clash [2 

on > a 

=> 

i is approved 

—an > a 


i is disapproved 


If a proposal is both sampled and relevant, than it is prior¬ 
itized. The coherent values of on = 0.5 (or 1/3) and 77 = 0.5 
were chosen as standards of the decision model. These are 
likely to change with implementation and management. 

Thresholds might be dependent on proposal, such as given 
by meaningful expressions. Immediate examples are: 


max({r)j, V/}) 
of = 1 - 7; 

These bonds among proposal variables has been discarded by 
the staff in the present initial steps. 


• adequate sampling of individual proposals and overall rank¬ 
ing. 

• Registration of the IP address and time of votes to ease 
detection of automated and other fraudulent efforts. 

• Reasonable use of the outcomes from the voting process. 
This requires probing the survey being conducted and its 
purposes. The indexes here presented target indicatives for 
the Brazilian federal government about the most important 
health care proposals. Given the unauthenticated voting, 
the outcomes might be regarded as reference rankings if 
data is minimally shared and checked for inadequate data 
entry (such as voting by automated scripts or a persistent 
participant introduced bias). 


http: / / pypi.python.org/pypi / percolate 
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Selected decision framework examples. Many of the online de¬ 
cision processes conceived and practiced resemble our model 
and have similar measurements to the at and 7 i indexes. This 
section presents a collection of models more familiar to the 
Brazilian participatory community, with focus on the mecha¬ 
nisms, not on historical notes. 

Pairwise [3] is part of the tackled paradigm: the ranking pro¬ 
cedure accepts new proposals while the voting occurs. Even 
so, pairwise voting is comparative, voter chooses between two 
proposals at each vote, and this does not fit proposed proce¬ 
dure. 

Appgree software [5] ranks proposals by sampling voters in 
cycles, each with fewer proposals. This is adequate for a range 
of decision making cases and showcases statistical estimates 
utility. The system has a separate proposition phase, and re¬ 
lies on an organized group engagement and user identities, 
which also does not fit current needs. 

Liquid Feedback [6] is a very renowned and bleeding edge 
solution for collective decision making. It relies on delegat¬ 
ing your voting count on specific subjects to other people you 
know or trust. Therefore, it does not fit current needs. Even 
so, this framework have precious considerations for our case, 
such as about ranking and presenting proposals to voters in 
the most useful ways. 

A Brazilian solution, used in diverse software and specially 
important as the output of a nation-wide decision making 
need, is the Agora Algorithm [7]. It presents a decision proce¬ 
dure in phases (agenda proposition, deliberations proposition 
and commenting, voting) with resolution outcomes. Although 
coherent, this framework requires authentication and might 
need experimentation and tuning in order to be effective with 
more than dozens or a few hundreds of participants. 

There is a number of other solutions for online collabora¬ 
tive prioritization, such as IdeaScale, Kidling, or any flavor of 
an Analytic Hierarchy Process (AHP). Authors hope to bet¬ 
ter formalize possible solutions (and found implementations), 
maybe through recommender systems theory [8j. 

1. Issue #19 for implementing priorization algorithm, 

https://gitlab.com/participa/proposal-app/issues/19 

2. Issue #29 for implementing priorization algorithm, 

https://gitlab.com/participa/proposal-app/issues/29 

3. Table developed for discussions about approval and participation indexes. Ronald Costa, 
https://docs.google.com/spreadsheets/d/lQ2uXcuKY0uL0nJdpjXoUaQhjmj 6mNv2St7F-pBi_31w 

4. Wiki surveys: Open and quantifiable social data collection, Salganik, Matthew J and 
Levy, Karen EC, arXiv preprint arXiv:1202.0500, http://arxiv.org/abs/l202.0500 

5. Appgree software for collective ranking of proposals, http://www.appgree.com 


Discussion 

The above estimates are the best fit the researchers could de¬ 
liver, suitable for current needs and not found (yet) in liter¬ 
ature. The following questions should be addressed in near 
future: 

• Are there more adequate metrics for ranking proposals in 
the given setting? 

• What are strong and weak aspects of the approach for col¬ 
lective recommendation? 

• What thresholds will be the choice of community and will 
they be adjusted with time? 

• Are there really no previous formalized model of this set¬ 
ting? If there is, what comparisons can we make on design, 
metrics and outcomes? 

• To which extent will participation community and public 
managers legitimize this approach? 

• What is the impact of this technological approach in public 
health care, social participation and the scientific commu¬ 
nity? 

• To which extent society benefits from this continuous vot¬ 
ing process? Is it worth the time spent by voters? How 
to evaluate this relation in terms of spent and gained re¬ 
sources? 

Most importantly, this report is being delivered to the civil 
society and the scientific community for consideration. Given 
the large number of possibilities for the collective ranking 
procedure, and the proliferation of solutions, research efforts 
might aim the organization of such procedures. 
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